Information Retrieval and Text Categorization with Semantic Indexing

In this paper, we present the effect of the semantic indexing using WordNet senses on the Information Retrieval (IR) and Text Categorization (TC) tasks. The documents have been sense-tagged using a Word Sense Disambiguation (WSD) system based on Specialized Hidden Markov Models (SHMMs). The preliminary results showed that a small improvement of the performance was obtained only in the TC task. 1 WSD with Specialized HMMs We consider WSD to be a tagging problem. The tagging process can be formulated as a maximization problem using the Hidden Markov Models (HMMs) formalism. Let S be the set of sense tags considered, and W, the vocabulary of the application. Given an input sentence, W = w1, . . . , wT , where wi ∈ W, the tagging process consists of finding the sequence of senses (S = s1, . . . , sT , where si ∈ S) of maximum probability on the model, that is: Ŝ = argmax S P (S|W ) = argmax S ( P (S) · P (W |S) P (W ) ) ; S ∈ S (1) Due to the fact that the probability P (W ) is a constant that can be ignored in the maximization process, the problem is reduced to maximizing the numerator of equation 1. To solve this equation, the Markov assumptions should be made in order to simplify the problem. For a first-order HMM, the problem is reduced to solving the following equation: